Papers with Neural Networks
Deep Learning and Sociophonetics: Automatic Coding of Rhoticity Using Neural Networks (N19-3)
Copied to clipboard
| Challenge: | Automated extraction methods for vowels are available, but coding rhoticity has lagged behind. |
| Approach: | They use Neural Networks/Deep Learning to train a model on 208 speakers in Boston . they find that there is no reliable method for classifying r-dropping . |
| Outcome: | The proposed method trains a model on 208 speakers in Boston, Massachusetts. |
Injecting Relational Structural Representation in Neural Networks for Question Similarity (P18-2)
Copied to clipboard
| Challenge: | Recent years have seen exponential growth and use of web forums, where users can exchange and find information just asking questions in natural language. |
| Approach: | They propose to use Tree Kernels to learn a model on relatively few pairs of questions as gold standard (GS) predicting labels on a very large corpus of question pairs is also a useful approach, they propose . |
| Outcome: | The proposed model can learn more accurate models after fine tuning on GS. |
Analogy-Guided Evolutionary Pretraining of Binary Word Embeddings (2022.aacl-main)
Copied to clipboard
| Challenge: | Existing binary word embeddings are derived from pretrained real-valued embeddables through different simple transformations, which often break the semantic consistency and the “arithmetic” properties of the embedded words. |
| Approach: | They propose a genetic algorithm to learn binary word embeddings from scratch by preserving the semantic relationships between words and the arithmetic properties of the embeddables themselves. |
| Outcome: | Evaluating 16, 32, and 64-bit word embeddings on Mikolov’s word analogy task shows that 95% of the time, the best fit is ranked in the top 5 most similar words in terms of cosine similarity. |
Pivot Based Language Modeling for Improved Neural Domain Adaptation (N18-1)
Copied to clipboard
| Challenge: | Existing work on domain adaptation does not exploit the structure of the input text . PBLM can naturally feed structure aware text classifiers such as LSTM and CNN . |
| Approach: | They propose a model that integrates pivot-based and NN modeling in a structure aware manner. |
| Outcome: | The proposed model can naturally feed structure aware text classifiers such as LSTM and CNN. |
Assessing Quality Estimation Models for Sentence-Level Prediction (C18-1)
Copied to clipboard
| Challenge: | Using a relevant QE model is also very important in QE. |
| Approach: | They evaluate a wide range of advanced sentence-level Quality Estimation models including Support Vector Regression, Ride Regression and Bayesian Neural Networks. |
| Outcome: | The proposed models behave differently in evaluation settings depending on whether test data come from the same domain as the training data or not. |
ProtoInfoMax: Prototypical Networks with Mutual Information Maximization for Out-of-Domain Detection (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to detect out-of-domain (OOD) inputs are limited and lack data. |
| Approach: | They propose a new architecture that extends Prototypical Networks to process in-domain and OOD sentences via Mutual Information Maximization objective. |
| Outcome: | The proposed method significantly improves performance up to 20% for OOD detection in low resource settings of text classification. |
Non-Parametric Adaptation for Neural Machine Translation (N19-1)
Copied to clipboard
| Challenge: | Neural Networks trained with gradient descent are susceptible to catastrophic forgetting due to parameter shift during the training process. |
| Approach: | They propose a semi-parametric approach that relies on local phrase level similarities to retrieve neighboring phrases that are useful for translation even when overall sentence similarity is low. |
| Outcome: | The proposed approach performs well on a heterogeneous dataset with WMT, IWSLT, JRC-Acquis and OpenSubtitles. |
Low-Rank Updates of pre-trained Weights for Multi-Task Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Multi-task learning is a popular approach for learning with pre-trained models due to the complexity of the tasks and the challenges associated with fine-tuning large pre-train models. |
| Approach: | They propose a new approach for Multi-task learning which is based on stacking the weights of Neural Networks as a tensor. |
| Outcome: | The proposed approach achieves equivalent performance to the state-of-the-art on the general language understanding evaluation benchmark by training only 0.3 of the parameters per task while not modifying the baseline weights. |